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1 .  Introduction 

The  present  paper  is  an  outgrowth  of  comments  made  by  the  writer  as  an 
official  discussant  of  the  stimulating  paper  [4]  by  Milton  Sobel  and 
George  Weiss  presented  at  Che  Nonparametric  Conference.—^  In  those  sections 
of  their  paper  in  which  3-player  tournaments  are  considered,  Sobel  and  Weiss 
formulate  their  problem  in  such  a  way  that  only  one  of  three  decisions  is 
permissible  after  the  tournament  is  completed,  i.e.,  one  must  choose  one  of 
the  decisions  '*Player  1  is  best,"  "Player  2  is  best,"  "Player  3  Is  best."  As 
the  writer  pointed  out  at  the  time,  this  foriulatlcn  of  the  problem  may  perhaps 
be  a  reasonable  one  in  some  situations  (e.g.,  when  the  tournament  director  is 
told  that  a  prize  must  be  awarded  to  one  of  the  players);  but  it  would  not 
be  appropriate  if  one  permits  the  decision  that  there  is  no  clear-cut  "best" 
player  in  the  tournament  —  to  cope  with  those  situations  for  which  the 
possibility  exists  that  (say)  Player  1  is  better  than  Player  2,  Player  2  is 
Detter  than  Player  3,  and  Player  3  is  better  than  Player  1.  (The  failure 
to  provide  for  this  contingency  causes  some  of  the  Sobel-Weisa  sequential 
Procedures  (those  described  in  their  Section  6,  and  which  are  based  on  the 
likelihood  ratio  statistic)  to  require  an  arbitrarily  large  number  of 
observations  to  terminate  sampling,  when  the  true  state  of  affairs  is  such 
that  the  probabilities  that  i  beats  J,  chat  J  beats  k,  and  that  k 
beats  i  (Ij^J,  J»*k,  k?*!,  l,j,k"l,2,3)  all  approach  one.)  The  formulation 
which  we  propose  in  the  present  paper  (and  which  seems  to  us  to  be  a  very 
natural  one)  permits  a  foui th  decision  "no  player  is  best,"  and  is  thus 
Capable  of  dealing  with  such  a  possibility. 

The  present  paper  has  two  main  parts.  In  the  first  part  (Section  2) 
we  formulate  our  ranking  problem  in  much  the  same  way  as  Sobel-Weiss  [4) 

— ^First  International  Symposium  on  Nonparametric  Techniques  in  Statistical 
Inference  held  at  Indiana  University,  Bloomington,  Iitdiana,  June  1-6,  1969. 
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do  theirs,  except  that  we  add  the  possibility  of  a  fourth  decision  to  their 
three  decisions.  We  add  a  probability  requireoent  to  their  probability 
requirement,  and  propose  a  single-stage  procedure  wh.'.ch  will  guarantee  both 
requirements.  A  numerical  example  is  provided.  The  indifference-zone 
approach  employed  in  [4]  and  in  our  Section  2  is  similar  in  spirit  to  the 
approach  used  earlier  by  the  authors  ([1),  [2])  for  certain  classes  of 
ranking  problems,  although  the  present  problem  is  quite  different  in 
structure  from  the  ones  which  we  considered  earlier. 

In  the  second  part  (Section  3)  we  formulate  an  Id'^ntif ication  problem 
which  also  parallels  the  one  formulated  by  Sobel  and  Weiss,  except  that 
(as  with  the  ranking  problem)  we  add  the  possibility  of  a  fourth  decision, 
and  an  additional  probability  requirement.  We  propose  a  sequential  Identi¬ 
fication  procedure  which  will  guarantee  both  requirements.  Incorporated 
into  our  procedure  is  a  certain  prior  probability  distribution  which  is 
assumed  to  be  known.  The  theory  underlying  the  sequential  Identification 
procedures  in  both  the  Sobel-Welss  paper  and  the  present  paper  is  developed 
completely  in  |2J  although  some  minor  modifications  are  necessary  to  Insure 
that  our  procedure  guarantees  our  two  requirements. 

Our  main  interest  really  centers  In  the  formulation  of  the  ranking 
problem  of  Section  2,  and  in  s^uenrial  procedures  which  guarantee  the 
associated  probability  requirement.  Our  identlf Ication  problem  of 
Section  3,  and  the  sequential  procedure  which  guarantees  the  “spoclited 
probability  requirement  can  be  regarded  as  a  first  step  toward  finding  a 
sequential  procedure  for  the  ranking  problem.  The  relation  between  such 
Identification  and  ranking  problems  is  discussed  in  Section  4.  Finally, 
a  more  detailed  consideration  of  certain  aspects  of  the  relation  between 
the  Sobel-Welss  paper  and  our  paper  is  given  in  Section  5. 


Sobel  and  Weiss  proposed  several  intuitively  appealing  sampling  rules 
for  their  sequential  Identification  procedure.  The  Monte  Carlo  s£impling 
results  which  they  obtained  using  these  rules  are  very  striking,  and 
demonstrate  that  this  la  a  very  fruitful  area  for  future  research.  Their 
sampling  rules  can  be  incorporated  in  our  sequential  identif l:uttlon  pro¬ 
cedure,  and  we  would  auticlpate  that  the  performance  of  our  procedure 
would  thereby  be  improved. 


f 

I 


JJW-Uli? 


2.  A  ranking  problem 


Let  •  P{Player  1  beats  Player  j}  1,J*1,2,3), 

(0  ^  ^  1);  the  pj^j  are  unkoovm,  and  It  Is  also  assumed  that  we 

have  no  knowledge  as  to  whether  p^^  <1/2  or  p^^  >  1/2.  (We  assume 

that  the  possibility  p^^  “1/2  cannot  occurV)  If  p^^  >1/2  we  say 

that  Player  1  la  better  than  Player  J.  Let  w  .  .  ,  be  the  State 

^  ab,cd,ef 

of  Nature  In  which  p  >  1/2,  p  ,  >  1/2,  p  -  >  1/2.  There  are  then  8 

ab  •^cd  ef 

possible  States  of  Nature  23  31’“  "’‘^21  32  31  group  into 

sets  0q,...,Q^  as  In  Table  I;  the  Interpretation  of  each  Is  also 

given  in  the  table. 


Table  I 


1  Relation  of  p. , 

to  1/2 

Interpretailon  of 

uc  xjL  vui  c 

‘’12 

^23 

•’ll 

"0 

“12,23,31 

> 

> 

< 

No  player  Is  best 

“21,32,13 

< 

< 

> 

“12.23,13 

> 

> 

> 

Player  1  la  best 

“12,32,13 

> 

<  . 

> 

1 

"2 

“21,23,13 

< 

>  j 

1 

> 

Player  2  Is  best 

“?1,23,31 

< 

> 

< 

“12,32,31 

> 

< 

^  < 

< 

Player  3  Is  best 

“21,32,31 

< 

< 

< 

1/ 


-  1/2  represeutB  the  boundary  reglonn  between  the  Q^-reglons 

given  in  Table  I. 
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2. 1  Goal,  and  probablllcy  requlranent 

The  goal  (purpose  of  the  experlaent)  la  stated  belov  aloag  id.tb  an 
associated  probability  requlreaent  (R). 

Goal:  We  wish  to  determine  which  one  of  the  (1>0,1,2,3)  represents 
the  true  State  of  Nature. 

We  permit  four^  possible  terminal  decisions.  These  are  given, 
along  with  their  Interpretation,  In  Table  II. 


Table  IT. 


Decision 

, 

Meaning  of  decision 

^0 

No  player  Is  best:  Either  Pj^,  >  1/2,  p^  >  1/2,  p^^^  <  1/2, 

or  Pj2  <  1/2,  P23  <  1/2,  Pjj  >  1/2. 

^1 

Player  1  Is  best:  Either  Pj^^  ^  1/2,  Pjj  >  1/2,  Pj^^  >  1/2, 

or  Pj^2  ^  1/2,  P23  <  1/2,  Pj3  >  1/2. 

Player  2  Is  best:  Either  p^^  -  1/2,  Pjj  >  1/2,  Pj^  >  1/2, 

or  1/2.  P23  >  1/2,  Pj^3  <  1/2. 

Player  3  Is  best:  Either  p^^  >  1/2,  p^^  <  1/2,  p^^^  <  1/2, 

or  Pj2  *  1/2,  P23  <  1/2.  P^3  <  1/2. 

~  It  Is  also  possible  to  consider  an  alternative  goal  which  It  "To  deter¬ 
mine  which  one  of  the  8  w's  represents  the  true  State  of  Nature." 

There  would  then  be  8  possible  terminal  decisions.  Corresponding  changes 
would  have  to  be  made  In  the  probability  requirement  (P). 


Prior  to  the  start  of  play  we  specify  four  constants  {pg,  Pq;  Pj^,  P|) 
with  1/2  <  Pq,  <  1;  1/4  <  P*,  P*  <  1  which  are  Incorporated  into  the 


0*  1 


following  probability  requirement. 


Probability  requirement  (R) : 


We  require  that  our  procedure  guarantee  that 


Pq  •  P{Malcing  decision  dQ)  ^  P* 


(2.1)  Rq:  when  mlnlp^^,  p^^,  p^^}  >  Pq 


when  minlpj^,  p^^*  Pi3^  -  Pq’ 


(2.2)  R, 


P{Makiog  decision  d^)  i  Pj 


when  min{p^j,  p^j^}  ^ 


(i-1.2,3) 


Jj^k;  J,k-1,2,3), 


2.2  Rfc;’lonc  of  preference  and  Indlf f '^r-’nce 


To  assist  the  reauer  in  visualizing  the  probability  requirement,  we  give 
a  geometric  interpretation  of  the  regions  in  the  parameter  space  where 
particular  decisions  are  preferred,  and  the  region  of  indifference.  The  true 


State  of  Nature  can  be  represented  as  a  point  p  ■  ^^32’  P23’  Pl3^ 
unit  cube  (0  ^  p^^^  ^  1),  (1  <  J;  1,J  ■  1,2,3).  For  simplicity  of  represent 

atlon  we  let  ”  1/2),  and  transform  the  parameter  space  into  the 

cube  (-1  <  Y  <  1),  (1  <  j,  1,J  ■  1,2,3)  which  is  depicted  In  Figure  1. 
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Then  In  Figure  1  is  the  rtglon  in  the  parameter  space  of  the 
where  decision  (i  ■  0,1,2,3)  is  preferred.  The  region  not  included 
in  the  region  of  preference  for  any  d^^  is  referred  to  as  the  region  of 
Indifference. 


2.3  Single-stage  procedure 

There  maj  be  many  statistical  procedures  (single-stage,  two-stage, 
sequential)  which  will  guarantee  the  probability  requirement  (R)  of 
Section  2.1.  He  describe  here  a  simple  single-stage  one. 

SINGLE-STAGE  RANKING  PROCEDURE 

Conduct  a  c-cycle  tournament,  a  cycle  consisting  of  3  games 
{Player  1  vs.  Player  2,  Player  2  vs.  Player  3,  Player  1  vs.  Player  3} 
where  c  is  a  predetermined  number,  computed  as  described  in  Section  2.4. 
Calculate  pj^2.  P23»  Pi3  where  •  (proportion  of  games  in  which 

Player  i  beat  Player  J  in  the  c-cycle  tournament) .  Compare  each  of 
these  p^j  to  1/2,  and  make  the  obvious  terminal  decision,  e.g.,  make 
decision  dg  if  p^2  ^  1/2.  P23  "  1/2.  <  1/2  or  if  p^^  "  l/^. 

A  A 

P23  <  1/2,  Pj^^  >  1/2.  Randomize  between  decisions  if  ties  occur,  e.g., 
randomize  between  decisions  d^  and  d^  with  probability  one-half  if 
Pj^2  ■  1/2.  P23  ^  1/2.  Pt3  ^  1/2.  ties  cannot  occur  if  c  is  odd. 

2.4  Determination  of  number  of  cycles  (c) 

Our  problem  now  is  to  determine  the  smallest  value  of  c  that  will 
guarantee  R.  For  simplicity  we  consider  only  odd  c  (and  thereby  avoid  the 
necessity  of  discussing  ties).  Denoting  the  cumulative  binomial  probability 
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Pr{X  ^  X  I  n,p}  =  I  y [  (nlyj ^  t>y  b(x|n,p),  we  have  that 


(2.3)  Pq  -  c  ,pj^2)  ■  c,p23)  ‘  c.p^j^) 


+  (I  -  B(^  c.p,^)l[l  -  B(^  c,p,,))(l  -  B(^  c.P3^)]  , 


’  I'  “V  2  '  2  |  '^23 

and  for  J»ti,  kf*!,  Jj<k;  J,k-1,2,3  we  have 


pX 1  1 

(2.4)  F.  -B(^ic.p  )  •  B(V‘''Plk^  (1  =  1. 2. 3) 


Since  Pq  >  1/2  we  have  that  Pq  is  minimized,  subject  to  min{p32.  P23»  Pll^ 

>  Pq,  when  pj^2  ”  P23  *  P31  “  Pq’  subject  to  max{p32.  P23»  P3i^  1  "  Pq 
(which  is  equivalent  to  Jiin{p2j,  P32*  Pi3^  =  Pq^  when  P32  “  P23  "  P31  “  1  "  PqI 


“■-‘^Pij.  Pii,'  r  Pi.  wneu  p^j  -  P^. 

P12  “  P23  *  P31  "  Pq  P12  "  P23  "  P31 

for  Rq  wfen  rainlp^^.  P23.  P31I  ^  Pq 
respectively,  and  the  configurations  p^j 
°'lu{Pij,  p^j^}  4  P3.  Thus  if  we  choose  Cq 
integers  c  which  satisfy 

(2.5)  c.Pq)  +  (1  -  B(^^-  c.Pq)) 


‘  P12  “ 

P23  “ 

P31  ■  Pq 

=1,2,3) 

is  minimized,  subject  to 

Pl-  We 

ref  er 

to  the  configurations 

P3I  ‘  ^ 

-  Po 

as  least  favorable  (LF) 

or  when 

max 

(Pl2>  P23'  P3i^  i  1  "  Pq 

'ij  “  Pfk 

■  Pl 

as  LF  for  when 

Cq  and 

‘^i 

as  the  smallest  (odd) 

respectively,  we  see  that 


(2.7)  c  -  max{cQ,  Cj^} 


Is  the  smallest  (odd)  Integer  which  guarantees  R. 


2.5  Numerical  example 

Suppose  that  the  experimenter  sets  up  the  specification  ■  0.60, 

Pq  •  0.75;  Pj^  •  0.65,  P*  ■  0.90.  We  find  using  (4)  that 

[B(22|43,0.60)]^  +  [1  -  B(22|43,0.60)J^  -  (0.908676)^  +  (0.091324)^  - 

0.751048  while  [B(21 | 41 ,0. 60) ] ^  +  (1  -  B(21 | 41 , 0. 60) ) ^  -  (0-903483)^  + 

(0.096517)^  ■  0.73840;  hence,  Cq  ■  43.  Also,  [B(15l  29,0, 65)  •= 

(0.952363)^  -  0.90700  while  (B{14| 27,0.65) »  (0.946377)^  -  0.89563; 

hence  c^  ■  29.  Thus  c  ■  max{43,29)  ■  43.  But  when  c  =  43  we  also 
obtain  ;?(22|43,0.65)]^  -  (0.978598)^  -  0.95765,  (B(22 |43,0.64) ] ^  - 
(0.970264)^  -  0.94141,  (B(22 | 43 ,0.63) ) ^  -  (0.959510)^  -  0.92066, 
(B(22|43,0.62) -  (0.945923)^  -  0,89477.  Thus,  for  no  extra  cost  in 
terms  of  sampling,  l.e.,  using  c  -  43,  the  experimenter  could  have  set 
up  and  guaranteed  stricter  probability  requirements  R  with  specifications 
such  as  (Pq,  pJ;  p^,  P* }  -  {0.60,  0.75;  0.65,  0.95765) . 


(0.60,  0.75;  0.63,  0.92066). 


t 
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3.  An  Identification  problem 

Let  be  defined  as  in  Section  2.  We  assume  that  p^,  Pj^,  0  are 

three  given  probabilities  with  1/2  <  p^,  p^^,  6^1,  and  that  there  are 
8  possible  States  of  Nature  23  31*  “2I  32  31  represent 

certain  possible  pairings  of  the  triple  (P22’  ^23*  ^31^  with  the  prob¬ 
abilities  Pq,  p^,  0.  These  pairings  are  given  in  Table  III,  as  are  the 
groupings  of  the  w^'s  into  sets  Q°,  ....  n°.  We  further  assume  a  known 
prior  probability  i  0  (  t(“°)  “  1)  associated  with  each 


one  of  the  8  States  of  Nature  In  problem  it  seems  reasonable 

i“0 

because  of  the  symmetries,  to  set  t(u)°)  -  (l-3A)/2  foi  each  of  the  2  uj°'s 
in  n®,  and  5(u°)  ■  X/2  for  each  of  the  6  uj'^'s  in  fi®  Then 

^0  "  ^^^0^  “  ^1  “  ^^“i^  •  ^  "  1,2,3). 


with  t’.ielr  interpretation,  in  Table  IV. 


t 
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Table  IV 


Decision 

Meaning  of  decision 

No  player  is  best  :  Either  pj^j  *  P23  “  P31  “  pQ’ 

or  pj,  .  P23  ■  P31  -  1  -  Po- 

‘^l 

Player  1  is  best  :  Either  “  P13  "  Pi»  P23  “ 

»r  Pj2  -  Pj3  -  Pj.  P23  -  1  -  «■ 

Player  2  is  best  :  Either  Pj^  “  P23  “  P^.  P32  * 

or  P33  -  P33  .  Pj,  P3,  -  1  -  e. 

Player  3  is  best  ;  Either  P22  ■  "  Pj*  P12  ' 

or  P33  -  P31  ■  Pj.  Pi2  ■  1  -  »■ 

Prior  to  the  start  of  play  we  specify  two  constants  {Pq  ,  with 

1/4  <  Pg  ,  P*  <  1  which  are  Incorporated  into  the  following  probability 
requirement . 

Probability  requirement  (R°) : 

We  require  that  our  procedure  guarantee  that 

(3.1)  R°  :  ^0  "  decision  when  is  true}  ^  P^ 

and 

(3.2)  ^2  •  ^1  ”  decision  d^  when  Is  true)  ^  P*  . 


(1  -  1.2,3) 
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(The  notation  of  (3.1)  and  (3.2)  is  Justified  by  the  fact  that  for  simplicity 
we  shall  consider  only  procedures  whose  symmetry  insures  that  for  each  1 
the  probability  expressions  of  (3.1)  and  (3.2)  are  the  same  for  both  members 
of  (i  -  0,1, 2, 3).)  It  should  be  noted  that  in  this  problem  we  are 

deciding  among  four  composite  hypotheses  (each  one  consisting  of  two 
completely  specified  States  of  Nature)  with  specified  lower  bounds  on  the 
probability  of  a  correct  decision  for  each  of  these  States  of  Nature. 


3.2  Sequential  procedure 

In  this  section  we  shall  describe  a  sequential  Identification  procedure 
which  will  guarantee  the  probability  requirement  (R°) .  The  theory  under¬ 
lying  this  development  is  derived  in  (2). 

Our  sequential  procedure  is  based  on  8  likelihoods,  each  one  of  which 
is  associated  with  a  different  one  of  the  eight  States  of  Nature  of  Table 
III,  These  are  defined  in  (3.3),  below.  In  making  these  deflnltlor.s  we 
assume  that  the  tournament  is  run  in  cycles  (as  defined  in  Section  2.3). 
However,  this  assumption  of  cyclic  play  is  made  only  for  con-' 
venier.ee,  and  the  same  results  would  hold  for  more  general  symmetrical 

playing  rules.  Let  s^^^^  -  (say)  denote  the  number  of  "succesies" 

m 

for  Player  1  when  he  plays  Player  J,  i.e.,  the  number  of  times  that  1 
beats  J,  in  their  first  m  games.  We  then  define  the  8  likelihoods 


(12,23,31) 


12  21  23  32  31  13 

8  f  •  X0  S  g  y  v8  0  g  ■%  v8 

Pq  ^^‘Pq^  Pq  ^^‘Pq^  Pq  ^^"Pq^ 


12,,^  23,  31  21  32^  13 

S  +8  H-8  ,,  .S  +8  +8 

Pq  ^^"Pq^ 


(21,32,13) 


m 


12  21  23  32  31  13 

/I  ®  /I  ®  /I  ® 

(I-Pq)  Pq  (l-Pp)  Pq  (I-Pq)  Pq 


21^  32^  13  12^  23^  31 

8  -t-S  -♦■8  , ,  .  B  +S  +8 

-  Pq  <1-Po> 


(3.3) 


21  23 


31  13 


^(12,23,13)  .  pS  98  ^j_p^j8  pS 


23  32 


31  13 


,(12,32,13)  8""  ,,  .s""  ..8^^  .8  ,,  ,8""  8 

L  ’  •  -  p  (1-p,)  (1-0)  0  (1-p,)  P, 


etc .  , 


and  the  4  likelihood  8ums 


(3.4) 


^  j  ,  ^^(12. 23,31)  ^  j  (21,32,13) 
tn  0'^  m  *in 

^  j  .  L^^2,23,13)  ^  ^^(12,32,13) 

ID  1  m  m 


L  (d,)  .  l<21.23, 31)  (21.23.13) 

m  2  tn  o 


Md  )  >  L<^2,32.31)  ^  ^^(21,32,31) 
m  3  m  m 


The  a  posteriori  probability  after  cycle  m  that  d^  Is  the  correct 

decision  given  ^  and  s  ■  is 

01  n  n  in 


(3.5)  P, 


L  (dj 


L  ^k 


(j  -  0,1, 2, 3). 


The  stopping  and  terminal  decision  rules  of  our  sequential  procedure  are 

based  on  the  P  of  (3.5)  and  on  a  constant  P*(l/4<  p*  <  1)  which  la 

J“ 

chosen  to  guarantee  (3.1)  and  (3.2).  As  stated  In  the  preliminaries  of 
Section  3,  we  shall  let  Cq  ■  1  -  3A  and  ^  (1  ■  3f2,3)  where  A 

Is  assumed  to  be  known. 


p 


SEQUENTIAL  IDENTIFICATION  PROCEDURE 

Sampling  rule:  Run  the  tournament  in  cycles. 

Stopping  nile:  Stop  the  tournament  at  the  first  cycle  n  for  which 

(3.6)  max  P?  >  P* 

0<)<3  J" 


where 


f 


for  X  »  0 


(3.7)  P*  -  /  max{pj  +  3X(1-Pj),  1  -  3X(l-pJ)}  for  0  <  X  <  1/3 


Pj^  .  for  X  ■  1/3. 

k 


Terminal  decision  rule:  Make  the  decision  which  has  associated  with  it  the 

largest  of  the  P°  .  If,  because  of  equalities  among  the  ,  several 
Jn  Jn 

decisions  are  associated  with  the  same  maximum  value,  then  select  one 
such  decision  by  a  random  device  which  assigns  the  same  probability  to 
each. 


The  terminal  decision  rule  can  be  stated  equivalently  as  follows. 
Letting  d  be  the  decision  d.  for  which  L  (d . )  -  max  L  (d.),  we: 

J  J  n  j  n  J 


Choose 

"o 

if 

^n^^O^  1-3X 

(3.8) 

Choose 

^3 

if 

‘  l-3k 

Randomize  among  ties. 


In  the  next  section  we  show  -liy  P*  of  (3.7)  guarantees  (3.1)  and  (3.2). 
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3.3  Determlnatlog  of  stopping  coastant  (?*) 

We  first  remark  that  in  general  it  is  difficult  to  determine  stopping 
constants  which  not  only  guarantee  probability  requirements  such  as  R  , 
but  also  do  so  in  some  optimal  sense.  This  ij  particularly  the  case  for 
nonsy metric  situations  when  the  number  of  terminal  decisions  which  the 
statistician  is  permitted  to  make  is  greater  than  two  (which  is  our 
situation).  In  fact  almost  all  of  the  theory  contained  in  [2]  which  deals 

with  sequential  identification  procedures  is  concerned  with  symnetrlc 
3lt<iations.  However,  it  is  possible  to  make  use  of  certain  results  and 
remarks  in  (2)  (specifically  those  In  Section  3.7(c),  pp.  51-52)  to  obtain 
solutions  for  certain  particular  problems  involving  nonsymmetrlc  situ-stlons. 
No  claim  is  made  that  these  are  the  best  possible  solutions  (which  they 
certainly  are  not),  but  only  chat  these  solutions  guarantee  (3.1)  and  (3.2). 

It  follows  from  results  proved  in  (2)  (specifically  from  Lemma  3.1.1 
and  Corollary  3.1.1  —  see  also  equation  (3.7.2))  that  our  sequential  identi¬ 
fication  procedure  of  Section  3.2  using  P*(l/4  <  P*  <  1)  on  the  r.h.s.  of 
(3.6)  guarantees 

(3.8)  3A  P°  +  (1  -  3X)P°  >  P* 

where  P°  and  “  ^2  ”  ^3  defined  in  (3.1)  and  (3.2),  respectively. 

From  (3.8)  it  follows,  by  setting  first  Pj  and  then  P^  equal  to  one, 

Chat 

(3.9)  P^  ^  (p*  _  3A)/(i  _  3x)  fQj.  0  <  A  <  1/3 


19 


(3.10)  P°  >  (p*  -  1  +  3X)/3A  for  0  <  X  ^  1/3. 

•♦r 

Since  for  fixed  X  the  r.h.s.  of  both  (3.9)  and  (3.10)  are  increasing 
functions  of  P*  which  approach  one  as  P*  approaches  one,  It  follows  that 
we  can  choose  P*  sufficiently  large  that  the  r.h.s.  of  (3.9)  Is  >  P*  and 
the  r.h.s.  of  (3.10)  la  i  P*  .  if  w®  let  P*  denote  the  smallest  value  of 
which  accomplishes  this  dual  objective,  we  see  that  P*  is  given  by  (3.7). 

3. A  Numerical  example 

In  Table  V  wr  glvo  the  values  of  P*  associated  with  the  specification 
P*  ■  0.75,  P*  ■  0.90  for  selected  values  of  1  “  1*2.3). 


Table  V 


X 

P* 

P*  -  3X  1 

r*  -  1  +  3x 

1-3X 

3X 

1/3 

0.90 

— 

0.90 

0.33 

0.9975 

0.75 

0.9975' 

0.30 

0.975 

0.75 

0.9722 

5/21 

13/14 

0.75 

0.90 

0.20 

0.94 

0.85 

0.90 

0.10 

0.97 

0.9571 

0.90 

0.01 

0.997 

0.9969 

0.90  ] 

0 

0.75 

0.75 

1 
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We,  note  that  when  P*  is  considered  as  a  function  of  X  for  fixed 
{Pq  ,  it  ffl  discontinuous  at  X  ■  0  and  X  ■  1/3,  e.g.,  P*  1  as 

X  -♦  1/3,  but  P*  ■  0.90  when  X  •  1/3.  However,  this  is  to  be  expected: 

As  X  -*■  1/3,  the  a  priori  probability  (,q  associated  with  0^  approaches 
zero.  But  the  procedure  is  still  required  to  make  the  decision  at 

least  75  percent  of  the  time  when  £1°  occurs  (however  Infrequently),  as  well 
as  to  make  the  decision  d^  at  least  90  percent  of  the  time  when  £1°  occurs 
(1  •  1,2,3).  This  will  in  general  require  many  cycles  to  terminate  the 
tournament  (and  hence  Che  expected  ntjober  of  cycles  to  texmlnste  the 
experiment  will  be  very  large).  But  when  X  •  1/3,  £J°  cannot  occur,  and 
the  procedure  la  only  required  to  decide  among  12^,  ^1^  and  make  the 

decision  d^  at  least  90  percent  of  the  time  when  £2°  occurs  (1  ■  1,2,3). 

A  similar  analysis  holds  for  X  0  and  X  ■  0. 

In  Table  V  the  minimum  value  of  P*,  namely  13/14  ■  0.9286,  occurs 
when  X  “  5/21,  In  general  this  minimum  value  occurs  when 
X  -  I  -  (l-P*)/3(2-P*-P*)  and  is  associated  with  P*  -  (P*  -  3X)/(1-3X) 
and  P*  -  (P*  -  1  +  3X)/3X. 

We  note  that  for  X  “0.33  (say),  the  experimenter  could  set  up  the 
stricter  specification  {P*  ■  0.75,  PJ  -  0.9975),  and  he  can  guarantee  it 
with  the  same  amount  of  sampling  as  ia  required  by  the  present  specification; 
such  a  tightening  of  the  specification  is  possible  for  all  X-values 
(0  <  X  <  1/3)  except  X  -  5/21. 

A  non-Bayeslan  might  regard  X  (0  4  ^  i  1/3)  as  a  constant,  at  his 
disposal,  to  be  set  In  such  a  way  as  to  cause  the  procedure  to  have  certain 
properties  which  he  deems  desirable.  Thus  he  might  set  X  ■  X,  and  still 
be  assured  that  the  procedure  will  guarantee  (3.1)  and  (3.2), 


■^SUL 
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A.  Relation  between  the  ranking  and  Identification  pr6bli 

It  was  demonstrated  in  [2]  that  In  many  situations  ot  practical  interest, 
appropriate  sequential  procedures  are  more  efficient  than  L,ir"le-stage  pro¬ 
cedures  which  guarantee  the  same  probability  requirements,  l.e.,  the  sequential 
procedures  often  guarantee  these  requirements  with  a  smaller  expected  number 
of  observations.  With  this  fact  in  mind,  we  would  like  for  the  problem  of 
Section  2  to  find  a  sequential  ranking  procedure,  to  replace  the  single-stage 
ranking  procedure  of  that  section,  which  will  guarantee  the  probability 
requirement  (2.1)  and  (2.2).  As  a  first  step  in  this  direction,  we  set  up  an 
artificial  identification  problem,  and  displayed  in  Section  3.2  a  sequential 
identification  procedure  which  will  guarantee  the  associated  probability 
requirement  (3.1)  and  (3.2)  (which  corresponds  to  protecting  against  the 
least  favorable  configuration  of  (2.1)  and  (2.2)  with  the  addition  of  the 
nuisance  parameter  0),  (usually,  if  one  cannot  solve  the  identification 
problem,  there  is  little  hope  of  solving  the  corresponding  ranking  problem.) 

The  next  logical  step  In  tne  research  process  would  be  to  try  to  produce  a 
sequential  ranking  procedure  which  reduces  to  the  sequential  Identification 
procedure  when  the  parameters  are  in  the  least  favorable  configuration,  and 
which  for  an  arbitrary  configuration  of  the  parameters  guarantees  the  prob¬ 
ability  requirement  (2.1)  and  (2.2).  (This  is  the  same  general  approach  as 
was  used  in  [2].  See,  In  particular.  Sections  3.6  and  6.1.)  However,  we  wish 
to  emphasize  that  we  have  made  no  attempt  in  the  present  paper  to  produce 
such  a  sequential  ranking  procedure,  but  rather  leave  that  as  a  topic  for 
future  research. 

One  would  naturally  be  very  much  interested  in  the  efficiency  (measured 
in  terms  of  expected  number  of  cycles  to  terminate  the  tournament)  of  our 
sequential  identli  Icatlon  procedure  of  Section  3.2  relative  to  that  of  a 
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single-stage  identification  procedure  which  guarantees  (3.1)  and  (3.2). 

(Note  that  the  determination  of  the  c  of  (2.7)  does  not  involve  knowledge 
of  the  value  of  the  nuisance  parameter  6,  while  the  use  of  the  sequential 
identification  procedure  of  Section  3.2  does  involve  such  knowledge.)  However, 
we  remind  the  reader  that  the  sequential  identification  procedure  of  Section 
3.2  which  uses  the  P*  of  (3.7)  is  very  conferwstive  for  0  <  X  <  1/3  because 
of  the  crude  inequalities  (3.9)  and  (3.10)  used  in  deriving  It,  i.e.,  the 
procedure  will  actually  achieve  probabilities  of  a  correct  decision  under 
(7°  (i  -  0,1, 2, 3)  which  are  substantially  higher  than  the  specified  values 
Pq  and  P*.  Thus  it  may  be  somewhat  misleading  to  compare  this  procedure  with 
a  single-stage  identification  procedure  which  Is  set  up  to  guarantee  the 
same  probability  requirementc  (3.1)  and  (3.2),  unless  some  adjustment  Is 
made  for  "excess."  (See  [2],  Section  3.7(d)  and  (e).) 


5 .  Relation  between  the  Sobel-Helss  paper  and  the  present  paper 


The  Sobel-Welss  paper  (4j  describes  selection  procedures  for  tournaments 
with  2  or  3  players.  Their  formulation  of  the  problem  for  tournaments  with 
3  players  is  analogous  to  ours  except  that  they  never  permit  the  decision  d^, 
and  hence  they  consider  only  the  probability  requirements  of  (2.2)  and 

of  (3.2).  Ideally  they  would  like  all  of  their  sequential  procedures  for 
tournaments  with  3  players  to  guarantee  our  probability  requirement  Rj^  of 
(2.2)  (their  requirement  (1.1)).  However,  at  this  point  in  the  development 
of  the  theory,  none  of  their  procedures  has  been  proved  to  do  so. 


Thv  y  h  IV..-  proved  that  thvlr  procedure 

does  so  if  the  value  of  the  nuisance  parameter 


p  in  our  (2.2)  is 
J  ^ 


known.  Their  procedure  R„  is  shown  to  guarantee  our  probability  requirement 
R°  of  (3.2).  Their  procedure  R^  is  intended  to  guarantee  R^  of  (3.2),  it 


is  studied  using  Monte  Carlo  sampling,  but  no  analytic  results  are  obtained. 
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The  Sobel-Welss  procedures  R^^,  and  R^^^  guarantee 

our  R°  of  (3.2);  they  use  the  same  stopping  rule  and  terminal  decision 
rule  as  does  our  sequential  identification  procedure  of  Section  3.2  when  we 
let  X  ■  1/3  in  our  proceduie,  but  they  use  different  sampling  rules. 

(Their  procedure  ^9  identical  to  our  procedure  when  we  let  \  “  1/3.) 

Their  sampling  rules  could  be  used  with  our  procedure  of  Section  3.2  for 
arbitrary  X  (0  ^  X  ^  1/3)  to  guarantee  (3.1)  and  (3.2)  simply  by  redefining 
the  L  's  of  (3.3)  to  read  since  in  general  we  will  have 


m 


(ab)  (cd)  (ef) 


m  .  ’  m  . 
ab  cd 


ef 


<“ab’“cd'“ef> 

with  ^  m^j  j*  The  Improvements  in  N  values 


(see  the  tables  in  (5]  sunmarlzlng  the  results  of  Monte  Carlo  sampling 
experiments)  obtained  using  their  sampling  rules  may  very  well  carry  over  to 
our  procedure  for  arbitrary  X  (0  ^  X  ^  1/3),  and  this  possibility  is 
certainly  worth  investigating  further. 

It  should  be  pointed  out  that  a  major  drawback  of  the  Sobel-Welss 
3-declslon  formulation  is  that  E{n}  will  become  arbitrarily  large  for  their 
five  procedures  R^^.  R^^,  and  if  mlnlp^^-  P23’  ^31^  "  ^ 

or  min{p22^,  Pi^3i  P32^  ^  (since  this  means  that  d^  is  the  true  State 

of  Nature).  However,  our  procedure  will  tend  to  terminate  early  in  either 


of  these  situations,  and  make  the  decision  d^. 


6.  Directions  of  future  research 

It  would  be  desirable  to  continue  the  present  Investigation  to  find 
sequential  ranking  procedures  which  can  be  proved  analytically  to  guarantee 
the  probability  requirements  (2.1)  and  (2.2),  and  which  employ  some  of  the 
sampling  rules  (or  variations  thereof)  studied  by  Sobel  and  Weiss. 


f 
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Research  should  be  devoted  to  finding  improved  values  of  the  stopping 
constant  P*  of  (3.7)  in  order  to  cut  down  on  the  overprotection.  It 
would  also  be  interesting  to  generalize  this  approach  to  tournaments 
involving  more  than  3  players. 
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M.  Sobel  and  G.  Weiss  have  proposed  sequential  selection  (identification  and 
rank ing)  procedures  for  3-player  tournaments.  In  their  formulation  of  the  problem 
they  permit  only  one  of  the  three  terminal  decisions;  "Player  i  is  'best.'"  (i=l,2,3) 

In  the  formulation  described  in  the  present  paper,  a  fourth  decision  is  also 
permitted,  namely,  "There  is  no  clear-cut  'best'  player  in  the  tournament"- - to  cope 
with  those  situations  for  which  the  possibility  exists  that  (say)  Player  1  is  better 
than  Player  2,  PI i/er  2  is  better  than  Player  3,  and  Plaver  3  is  better  than  Player  1 
For  this  4-decisi<''n  formulation,  a  single-stage  ranking  procedure  is  proposed  which 
guarantees  that  certain  requirements  on  the  probability  of  a  correct  selection  will 
be  achieved  (no  matter  which  one  of  the  four  States  of  Nature  is  the  true  one);  for 
this  same  formulation,  a  sequential  identification  procedure  is  proposed  which 
guarantees  that  certain  more  restrictive  requirements  on  the  probability  of  a  correct 
selection  will  be  achieved.  Our  sequential  identification  procedure  can  be  regarded 
as  a  generalization  of  one  of  the  sequential  procedures  proposed  by  Sobel  and  Weiss. 

The  relationship  between  certain  aspects  of  the  Sobel-Weiss  paper  and  the 
present  paper  is  discussed  in  some  detail.  Directions  of  future  research  are 
proposed . 
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